Digital In-Context Experiments (DICE)
We need a new subtitle that does not exclusively focus on validity.
Hauke Roggenkamp 
Johannes Boegershausen
Christian Hildebrand
Case Studies
The following case studies demonstrate the practical application and novel capabilities of DICE. They not only showcase the tool in action but also highlight its key contributions, particularly by manipulating entire feed contexts and in measuring dwell time. By presenting these studies, we aim to provide a blueprint for researchers interested in adopting DICE for their own studies The first case study illustrates the tool’s capacity for manipulating and controlling entire feed contexts whereas the second focuses on measuring participant engagement through dwell times. Together, these studies exemplify how our tool can enhance ecological validity while maintaining high levels of internal validity as discussed above.
Context Matters: Evaluating Brand Safety in Social Media Advertising
Brand safety refers to strategies and measures ensuring that a brand’s content, particularly advertisements, does not appear in contexts that could harm the brand’s reputation (see, e.g., Bellman et al. 2018; Lee, Kim, and Lim 2021; Hemmings 2021). These measures are especially crucial in social media, where platforms use automated systems to place ads in dynamic, rapidly changing, and user-generated content environments. Such automated systems often lack the nuanced understanding that humans possess, which can lead to ad placements in contexts that seem appropriate at first glance but are ultimately unsuitable. In our hyper-connected world, such of misplacement can rapidly propagate, potentially magnifying reputational damage beyond the initial exposure (Swaminathan et al. 2020). Accordingly, Ahmad et al. (2024) found that most brand managers have a strong preference to avoid misplacement and Schmitt (1994, 1986) quotes an expert saying “Advertisers [do] not want to display their products between battle scenes.” This is also reflected in an industry report that not only states that about 70% of brands take brand safety seriously but also that 75% of the interviewed brands report brand-unsafe exposures (GumGum Inc. 2017).
Brands typically consider juxtapositions with hate speech, pornography, and violence as the most egregious violations of brand safety. To mitigate such risks, brands and platforms commonly employ blacklists and negative targeting strategies, defining keywords and publishers associated with these topics to avoid undesirable ad placements. On X (formerly Twitter), for instance, brand managers can utilize adjacency controls, allowing them to specify up to 1,000 negative keywords to regulate the content appearing above and below their ads in users’ timelines. While these measures have proven relatively effective in preventing placements alongside the most brand-unsafe content, misplacements adjacent to disasters, tragedies, divisive political content, and misinformation remain prevalent and, in part, unnoticed: Ahmad et al. (2024) find that most decision-makers are unaware that their companies’ advertising appears on misinformation websites. This persistent challenge may be attributed to the inherent difficulty in accurately classifying and identifying fake news, subtle forms of divisive content, and emerging crisis situations in real-time. GumGum Inc. (2017) reported that 39% of sampled brands experienced their content being displayed adjacent to at least one of these problematic topics.
To illustrate the unique capabilities of DICE, we propose a simple study that extends beyond altering individual posts to modifying entire feeds: Unlike traditional online platform studies, we hold the ad copy and creative constant while manipulating the surrounding context between-subjects. Importantly, this study design is uniquely feasible within the DICE paradigm due to its precise control over the contextual environment—a capability not available in other research methodologies such as vignette studies. This level of control is crucial when examining brand safety, a phenomenon inherently defined by an advertisement’s context. By manipulating the surrounding content while keeping the ad constant, we can directly investigate how context impacts brand perceptions, offering insights into brand safety that would be challenging to obtain through alternative research approaches.
We test the intuitive hypothesis that an inappropriate (compared to a more general) context negatively affects brand attitudes. To better understand whether the effect is also driven by implicit memory effects (Schmitt 1994), we control for cued and uncued recall.
Experimental Design
Our study focuses on scenarios where airlines promote travel destinations through targeted advertising, placing ads in contexts that align with specific destinations. Given that major airlines serve numerous destinations globally, these ad placements are typically managed through automated programmatic systems. We leverage this automated placement approach to create two hypothetical scenarios featuring KLM Royal Dutch Airlines (KLM) promoting flights to Brazil. At the time of the study, Brazil was experiencing severe flooding that claimed at least 95 lives (Buschschlüter 2024). To simulate real-world conditions, we scraped real tweets and assembled them to two distinct Twitter feeds: one covering the natural disaster and another featuring more general content, including coverage of Madonna’s free concert in Rio de Janeiro. This experimental design allows us to examine the impact of contextual advertising in varying circumstances, including during times of crisis.
For the illustrative character of this study, we assumed that automated placement systems primarily target the keyword “Brazil” without considering nuanced contextual factors. This assumption allowed us to simulate how the same advertisement might appear in markedly different contexts on a social media platform. Consequently, we placed an identical fictitious sponsored post by KLM, promoting flights to Brazil, into both Twitter feeds. The advertisement features a creative (as shown in Figure 1) as well as copy that read: “Brazil’s wild beauty calls! Experience nature like never before. Book your breathtaking adventure with KLM.” While this messaging would typically be considered appropriate for tourism promotion, it appears strikingly insensitive when juxtaposed against news of a natural disaster. [Shall we pre-test this assumption?]